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ABSTRAtT 

This paper 
deteraine the possible use 
■easure of achieveaent in a 
civilization class. Twenty 
Spanish class at' the Onirer 
•ultiple choice tests and c 
studied during th^ course^ 
vere highly significant and 
t^at the cloze test is a be 
cultural knowledge and is ■ 
choice reading test. 01;he, 
cloze procedure in the clas 



reports on a pilot study conducted to 
of the cloze procedure as a' substitute 

second-year Spanish culture and 
students enrolled in a third-seaester 
sity of Colorado were siaultaneously given 
loze tests on four cultural essays they 
CorrelatioRfil^ betw€en the two^nstrunents 

ranged froa ••32 to •67. It is concluded 
tter discriainator of the. acguisition of 
ucji^-^sier to construct than a multiple 
l^vabtages and disadvantages of using the 
srooB are discussed ^ (Author/AH) ^ 
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€3 '"The Cloze Proc^edure as a Progress Test"*^ 

OO During the past 25 years much , research has teeen done on 

^r— f . the cloze- procedure, or ^loze test as it' is more of'ten called 

\^ I ^ in the field of foreign language education, Taylor was th^ 

. ' fi^-st'to experiment with it, when in 1953, he istudied it as ^a 
I ^ * measure of contextual redundancy ^ in hi& dissertation at the 

University of Illinois. Taylor named the technicjue the cloze 
procedure /r deriving it from the concept erf. closure" in Gestalt 
' • psychology. GestaltWts believe that learning follows a se- ^ 
• ' quence thJ^ugh which one first understands, the wHole, or 

- broader ' issues,, and then^.grasps the individual details. Ap- 
plied to visual perception, the theory suggests that in look- 
ing at a tree?,' we first see it ^s a single unit, rather than 
as a collection of individual leaves and branches. Once this 
whole is perceived ,^e can then begin to make note of indivi- 
. , * dual details such as groupings of folia'ge ^missing branches. 

^ • Closure fefers to the natural tendancy to mentalJ^y close the 
gap or fill in the spaces, such as in .perceiving a not-quite- 
^ ' complete circle as a whole .circle*^ Likewise, the cloze -pro-' 

^ ^ * / endure' requires the students to perceive the whole, by filling 

0 ►in missing wqjjds, ^s'if they were nbt mi-s^ng at aU^.. 

The cloze procedure was originally used as a measure of 
f^the readability or difficulty of a prassage of prose. It is 
constructed by systematically eliminating every Nth word in 
a passage with the distance between^ deletions usually being 
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nei34rher less than five worcis nor jnore than ten*words^ 
The ejxaminee attempts to .reconstruct the pass^aige by fllj 
ling in the missing wordsi^ Taylor later discovered the 
cloze' coul^ serve' as a meaWre of pjior -familiarity with, 
or mastery of a specific body of knowledge. In a st.udy 
.-supported; by the^U.S. Air Force/he administered an 80- 
item cl^z^ test and a multiple choice comprehension test 
on the Air Force system of supply to three groups of ser- 
vicemen. One group- t?I=52) received an easy cloze test, 

r 

based only on- verb auxilaries, conjunctions, pronouns and 

articles. And1;her .group *(N=48T received a cloze- test of . 

moderate difficulty based on th^ systematic deletion of 

every fifth word. A third group"^ (N=52) ^receive^d\ har;^ 

cloze test which deleted only aclverbs, verbs, and nouns.- • 

<^ If 
Only tho^e , fill-ins which matched the original word we^^ 

counted as right. 'Following this pretestS the' subjects 

were aske^ to study a 3200 word technical article on the 

Air Force supply system. One week later th6* subjects 
I 

were retested. The pretest, served as a measure of prior 

km©^ledge and the post-test ferved as a measure of- know- ' 

ledge ax:quire'd fallowing instruction. JVhile all three 

groups showed ^significant gains on the post,-test, sco^s 

during both administrations correlated highly, witji Pearson 

coefficients ranging from^ .58 to .92. Taylor Gc5ncluded 

y ^ / ^ 

that cloze scores are influenced by pertinept^ specific 

knowledge and overall language facility. ' 

BetVeen 1957 and 1959^, Carroll, Carton, and Wilds con- 



^ducted S feasibility s-tudy on the possible inclusion of 
the cloZe procedure in the College^ Entrance Examination . 
Board's tor^ign language tests. ^ It may surprise research- 

^efs today tcr learn that the authors rejected the cloze pro- 



cedure as a prof iciency "Measure, . "because they are relativ- 

el^ unreliable ana\ too heavily affected by various sources 

\ ' . ^ 

of extraneous variakice" - While' not alv?ays clear on this 

matter^ the authors indicate the\soufces of extraneous 

variance are the exa^iaee's level of "general knowledge^ 

familiarity with the" subject, ability, to assimilate new 

information while^ completing the passage, and general .rea- 

seeing abidity*. Per^iaps because of this conclusion, and 

^because' of the instr.ument ' s lack of face uarlidity, a full 

decade passed beffltr^^ another published stAidy directed at- 

tentlon to the cloze procedure in the assessment of langlia 

proficiency. * - 

I 

In 1968, *^Darnell correlated scores obtained with .a 
modified cloze procedure called clozentropy, ^wrth-^scor^fe ; 
obtained on the^ Test of English as a Foreign Language pub'- 
lished by Educational Test ing . Service . Using 48 foreign 
students at the University of Colorado as subjects, -he ob- 
tained a c6rrelation of. .84 between the two instruments.^ 

Following publication of Darnell's findings, Bowen,^ 
Kaplan and Jones, ^ Oiler and Conrad;^ Stubbs and Tucker, ^ 
and many others iTiVestigated and confirmed the utility of 
this instrument j|Ln assessing overa 11 second language pro-^ 
ficiency. Other'^uses are yet^ to be discovered. >w 
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^The present study was intended as a jiilot project 

and presents the results of an informal classroom exper- ^ 

iment conducted by the author to investigate 'the - possible 

application of the qloze procedure ^s ^^p-eur'fogate measure 

of achievement in second year reading courses in a foreign 

-language. The ^author wanted to , ascertain' whether the degree 

of familiarity with a reading selection, would determine a^ 

student *s score on a cloze test. Because of Taylor's pre- 

Vaous finding, it^was- hypothesized th*Bt'there wouldj^e a 

high correlation between the s^cores obtained on multiple ^ 

choice reading tests and cloze passages taken from the 

reading seTetrtions studied by the olass. 

The following sentence illustrates Aow i^S^as antij- 

cipated the pr.ocess would woofftfT" ^ - ( , 

Once" a time there we;:e *little 

A / B ' ^ ' 

pigs. 

While the native speaker of English could probably write 
"upon" on linfe A,^he would not know that the answer t^B 
was "three" unless he was -familiar with the story. Other- 
wise, he coul.d be expected to fill in such words as "some", 
"these" ,^ or "several" A foreign language learner,^ lat:k-^ 
ing the native speaker's structural proficiency*, might miss 
both -words, unless very familiar with the story. 
Method. ■ During the summer of .1975, /the author taught "a 
third semester Spanish' ^course to ^20 ^ students at ^the Univer- 
sity of Colorado^. The curriculum was based on .the pr-elim- 
iriary edition of a review grammar ajq^. co-U.e;::tion of cul- 
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tural essays by Copeland/ Kite, and Sandstedt.'^ The 
cultural essays were oriented toward the social saiences- 
and dealt with various aspects of the IJispanic worjd such^ 
as family, death", religion,^ pre-ColumbiAn -civilizations , « 
and economic problems. Each selection 'was approximately' 
2000 words in length. 

% 

After rea^ding, discussing, an'd studying each essay, 
the students werre given a.lS^item, f our-option^ultiple^ 
choice test based bn the content Qf the selection. • Im-- 
mediately afterwards, they were giv^n two .paragraphs sel- 
ected ^rom thqiwpassaga and systematically mutilated ac- 
cording tfo the cloze procedure. The. two paragraphs sel- 
ected were from different parts of the essay ar)d , contained 
a total of 30 deletions for studerji:s to fill in.- Following 
each toting session, the two sets of scores were corre- 
lated ' (Pearson ' s r) on a hand calculator. - The obtained co- 
efficients are listefl below. 

TABLE I*^ " • , 



Product Moment Correlatiws Between Clozfe and 
Multiple Choice Scores' on Four Tests* 
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Test 


Corre lation 


Nth. Word -Deleted 


I 


. 32 


* 5thT^ 


II 


.67 


7ttv** . 


III 
IV • 


^.61 


9 th* 






«p< .|005 



> Resultg and. Analyses ; On the first cloze. test, Taylor's 

scheme of de^etin^ every f i f th' word . w^s followed, Thi's 

^resulted in a cellar effect in that thev highest score was 

only 18 of 30 items correct. This attentuated distribution 

would accCHijit for the low correlation between scores on 

this administration, since a reduction in variance will 

always lower the correlation between Scores that are less 

than perfectly reliable. On the second test every seventh 

word was del-eted. While this improved the -dis tributioi^/ 

most ^^Loze .scores were still less than 50% correct.^ On 

s 

^ the third and foufth tests, the ^^umber of words between 
each deletion again increased until on the' fourth test, 

the mean score was 68% correct. . Mean scores on the four 
multiple choice tests probably prevented the Correlation 
from reaching higher levels throughout the course. None- 
theless, the correlations on tests II, III, arid IV v^ere 
highly sign^if icant . 

Disd^s^sion > It is interesting to pondet the ifnpllcations 
of this pilot study. Although the correlations were sig- 
nificant,- the common variance c^an not be a^itributed ,to 
knowledge of the"^' content alone. It could^be that a gen- 

. eraL language^ pro'ftciency factor is responsible for simi- 
ikr resuPETg -on both tests.. . ' Un<Jbubtedly, this factor would. 

* always be responsible ' for some 'simi lar ity in scores * It . 
' would be interesting for future studies to include an un- 
familiar cloze passage as a genera-l proficiency control . 

. measure^ and then compare correlations. If the familiar 

.7 - ' ■ ' ' • 



H cloze correl&ted better with the unfamiliar cloze than- 
with the multiple-choice test/ then it would have little 
f . usefulness as a progress te§t. On the other hand/ "if itJS . 
' correlation with - the 'multiple-choice pj?t>gress test were 
^ substantially greater, then the cl|:)ze • could serve as ar\ ' 

easy-tOf construct* substitute Tiieasure. 
^ * Anotheir question which -merits the attention of future 

I research on this topic is the effect of 'selective rather 

. * > • N ' . 

than systematic deletions- Taylor f.ound that function 
\ ^ ' 

^ words (articles/ pronouns, auxilaries, and conjunctions) 
were much easier to replace than major content words (nouns 

• and verbs) / although the/ correlations obtained with -separate 
cloze tests of each did."-not exceed that obtained by sys- 
tematically deleting every ^ifth word.^ Nonetheless, the 
exclusion of function Vord^ from the passage^ mutilation pro- 
cess could produce a better test of kriowle'dge* acquisition 
for foreign languages since it might partially control for 
differences in language proficiency among learners. As 
Oiler demonstrated in his study comparing exact word and ' 

\ 

cont^xtually acceptable scoring methods, the findings of 
cloze research' based on native* speaking su^) jects • can • not be 
^^'"generalized to second language learners . 

Use of the cloze test in the classroom situati^on re- . 

* . ♦ i • 

r.- ^ ^ • suited in 'certain reactions which merit reporting. In 

general, studen€s disliked it and felt that the ins^uctor 

was testing^ the remembering* o^ Vords instead of the acqui- 

sition' of cultural knowledge, \a1so,, Binc^ the i^esults of 



±,h^ cldze 'test -were counted in determining the-'final gr'ade,^ 
students st.udied the e;&say assiduou4 LT^ntil they, not only 
knew its content/ but could remember in -what order idea3 
were' discussed. As a\reSultv the multiple choice reading 
tests' became very easy, in- spite of an attempt to make them 
demandifTg! ^and this prevented tfte 'obtaining of a reliable 
spread of scores/ On the other hand, the instructor was 
pressured to make the cloze te^t easier by a barrag^ of 
initial complaints. Therefore, on successive tests the 
distance ^between deletions was increased. Once the scoi^^s 
improved, complaints about the cloze subsided. ^ 

Use of the cloze test also affected students' reten- 
tion of vocabulary. Fallowing Taylor's procedure, only 
exact word which Occurred ;j.n the text was accepted; As^^^.-sr"""^ 
result, students made a greater than norma 1 ef^oix to ab- 
sorb new words into th^ir active vocabular^^ so that they 
could produce them on the cloze test. The same effect was 
also noticed for structure. . T^e cloze passages forced stu- 
ofents to use many structural items which- vould normally be 
above the producti^^e syntactic abilities of -second-year 
students. These included* contextually- appropriate relative 
pronouns, ^conjunctions, miscellaneous clause 'and sentence 
relators / and compound verb tenses . As a result , the use 
of the cloze 4iad a reinforcing ef^fect on the grammar review 
and ^it accelerated language acquisition in general. This is 
becalise "'the cloze prohibited the students from becomiji^* sat- 
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isfied with learning jusJb.the contejlt of the passage. It 
made^theitl acquire a more sqphisticated 'lev«l of language. 

• . , . r 

Instead of permitting a^ disparity between receptive and pro- 
ductive skills, it forced the -student to integrate these itito 
a single active system. ' ' ' - * 

In sugary., in my first experience vith the cloze pro- 
cedure as a progress test^ it performed better in its teach- 
ihg than its testing function.- While adverse student, at: - 
tltudes were initially. a problem, these were overcoiije with 
experience and practice. While the cloze did not ^how high 
correlations with the multiple choice t^st of content know- 
ledge, this was probably due to the latter' s lack of dis- 
crimination power. The* author felt that the cloze was 
functioned better as a measure of 'familiarity w^th the con- 
tent, while it encouraged students to improve their -overall^ 
proficiency. AlsOf it 'was much easier" to 'const;ruct and jus^ 
as easy to grade. Future studies; in both controlled ex- 
perimental and classroom settings, wi^l have to verify the ' 
procedure's utility as a progress test. For the present . 
however, the'se results seem encouraging. , ^ - 
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